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Abstract 

A real Hnite dimensional space with indefinite scalar product having V- negative squares and v+ 
positive ones is considered. The paper presents a classification of operators that are normal with respect to 
this product for the cases min{v-,v+} = 1, 2. The approach to be used here was developed in the papers 
^ and [^, where the similar classihcation was obtained for complex spaces with v = min{v- ,v+} = l, 2, 
respectively. 


1 Introduction 

Consider a real linear space i?" with an indefinite scalar product [• ,•]. By definition, the latter is a non¬ 
degenerate sesquilinear Hermitian form. If the ordinary scalar product (■ ,•) is fixed, then there exists a 
nondegenerate Hermitian operator H such that [x,y\ = {Hx,y) Va:,y G i?". If A is a linear operator 
{A : i?" ^ .R"), then the H-adjoint of A (denoted by is defined by the identity [A[*H,?/] = [x^Ay], An 
operator N is called H-normal if NN^*'^ = an operator U is called H-unitary if UU^*^ = I, where I 

is the identity transformation. 

Let U be a nontrivial subspace of R”. The subspace V is called neutral if [x, y] = 0 Vx, y G U. If from the 
conditions x G U and Vy G U [x, y] = 0 it follows that x = 0 , then V is called nondegenerate. The subspace 
UHl is defined as the set of all vectors x G R": [x,y] = 0 \/y £ V. If U is nondegenerate, then UHl is also 
nondegenerate and U-i-UH] = ij"-. 

A linear operator A acting in R" is called decomposable if there exists a nondegenerate subspace V C R" 
such that both V and UHl are invariant for A or (it is the same) if V is invariant both for A and AI*! . Then 
A is the orthogonal sum of Ai = A|y and A2 = A|y[j_]. If an operator A is not decomposable, it is called 
indecomposable. 

Throughout what follows by a rank of a space we mean v = min{u_,u+}, where V- (u+) is the number 
of negative (positive) squares of the quadratic form [x,x], i.e., the number of negative (positive) eigenvalues 
of the operator H. Note that without loss of generality it can be assumed that (otherwise H can 

be replaced hy —H; the latter (nondegenerate Hermitian operator) has opposite eigenvalues). Later on we 
assume that < u_|_. 

The problem is to obtain a complete classification for iL-normal operators acting in i?”, i.e., to find a 
set of canonical forms such that any iL-normal operator could be reduced to one and only one of these 
forms. Since it is sufficient to solve the problem only for indecomposable operators, for any nondegenerate 
Hermitian matrix H and for any indecomposable iL-normal matrix N we would like to point out one and 
only one of the canonical pairs of matrices {N,H} so that the pair {N,H} is unitarily similar to {N,H} 
(two pairs of matrices {iVi, Ri} and {N2, R2}, where Hi and H2 are nondegenerate Hermitian matrices, are 
called unitarily similar if N2 = T~^NiT, H2 = T*HiT for some invertible matrix T; if Hi = H2, then they 
are Hi-unitarily similar). In what follows such a classification is presented for operators acting in spaces of 
rank 1 and 2 . As in [2], we will denote by A the identity matrix of order r x r, by the r x r matrix with 
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I’s on the secondary diagonal and zeros elsewhere, and hy A (B B (B ■■■ (B C a block diagonal matrix with 
blocks A, B, ..C. 

We are grateful to Prof. Leiba Rodman for his attention to our work and very helpful comments on this 
paper. 


2 On Decomposition of i7-normal Operators in Real Spaces 

Let an iJ-normal operator N act in i?” and have p distinct real eigenvalues Ai, A 2 ,..., Ap and q distinct pairs 
of complex conjugate eigenvalues a^+i ± if3p+i,ap+2 ± 'i'Pp+ 2 , ■ ■ ■, cxp+q ± ifip+q- Let us define 

. ._r (A-Afc)”, if l<k<p 

Vk\ _ 2ak\ + , if P < k < p + q, 

Qij = {x : (pi{N)x = (pj{N^*^)x = 0}, i,j = l,...,p + q, 

^ = {(hj) ■ Qij 7^ {Oil- 

Proposition 1 The subspaces Qij have the following properties: (1) Qij n Qki = {0} V {i,j) {k,l). 

(3) Each subspace Qij is invariant for both N and (4) Eigenvalues of the operator N\q^^ are roots 

ofipi{X), those of the operator are roots ofipj{X). (5) [Qij, Qm]= 0 X{i,j)^{l,k). 

Proof: 

(1) Suppose (i, j) {k,l). Without loss of generality it can be assumed that i k. Let 3a; : x G Qij, 
X G Qki, i-e., (pi{N)x = (pkiN)x = 0 . Since the polynomials Pi{X) and (pk(X) are relatively prime, there exist 
polynomials tpiiX), tpkiX) such that the matrix identity I = ipi{A)(pi{A)+'(pk{A)pk{A) is valid. Consequently, 
X = ipi{N)ipi{N)x + iik{N)ipk{N)x = 0. 

(2) The greatest common divisor of the polynomials ^i(A) = Wi^iPiiX), £, 2 {X) = Y\i^ 2 Ti{^)^ 

fp+q{X) = n i^p+qTiW is equal to 1, therefore, there exist polinomials ^/’i(A), 'ip 2 iX), ..., ipp+qW such 
that I = J2i=i XA. Hence, \/x x = J2i=i '^iiN)fiiN)x = Xi (where Xi = 'ipi{N)^i{N)x). 

Since the product of all (pi{X) annihilates N, we have ipi(N)xi = 0 Vi, i.e., i?" = Xr=i where Qi = {x : 
(pi{x) = 0}. Similarly, each subspace Qi is a direct sum of the subspaces Qij = {x € Qi : (pj{N^*^)x = 0}. 
Disregarding the trivial subspaces Qij, we obtain the desired equality i?" = X(i j)Gn Qij- 

(3) Since N and iVl*! commute, for all {i,j) and x G Qij we have 0 = Nipi{N)x = (pi{N)Nx, 0 = 
N(pj{N^*^)x = (pj{N^*^)Nx, i.e., Nx G Qij. It can be checked in the same way that iVl*lx G Qij. 

(4) Let N\q^. have an eigenvalue Aq such that i^i(Ao) 7 ^ 0. Then there exists a (real or complex) 
eigenvector x 7 ^ 0 corresponding to the eigenvalue Aq. Since the polynomials A — Aq and (pi{X) are relatively 
prime, there exist polynomials i^iiX), ip 2 {X) such that the identity I = 'iIji{A){A — Aq/) + 11:2(A)(pi(A) 
holds for all (complex) matrices A. Consequently, x = 'tpi{N){N — XoI)x + 'tp 2 {N)Pi{N)x = 0 because 
[N — XqI)x = (pi{N)x = 0. The contradiction obtained shows that all eigenvalues of iVlg^ are roots of 
(pi{X). The operator iV[*lIQy can be considered in the same way. 

(5) Let i 1. Take arbitrary vectors x G Qij, y G Qki. Since the eigenvalues of fVIg^ are not roots 
of pi{X), the operator (pl{N)\Q^. is nondegenerate. Therefore, 3z G Qij : (pi{N)z = x. We have [x,y\ = 
[ipi{N)z,y\ = [ 2 ,(^/( 7 VW)y] = [z,0] = 0. 

The proof of the proposition is completed. 

Now let Vi = Qii {{i,i) G D), Vjk = span{Qjk,Qkj} {{j,k) G D, j < k). The subspaces Vi, Vjk are 
mutually orthogonal, the intersection of any two of them is zero, and their sum is BA. It follows from the 
nondegeneracy of El that each subspace V), Vjk is nondegenerate. The restriction IV|y. has the only real 
eigenvalue Ai if z < p or the pair of complex conjugate eigenvalues ai ± ifii if z > p. The restriction N\vj^ 
has two distinct real eigenvalues Xj, Xk if j, k < p, one real eigenvalue Xj and the pair of complex conjugate 
eigenvalues ak ± iPk A j < p, k > p, or two distinct pairs aj ± ijSj, ak ± iPk if j, k > p. 

Thus, we have proved the following lemma: 
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Lemma 1 Any H-normal operator N acting in is an orthogonal sum of H-normal operators each of 
which has one of the following sets of eigenvalues: 

(a) one real eigenvalue; 

(b) two distinct real eigenvalues; 

(c) two complex eonjugate eigenvalues; 

(d) one real and two complex conjugate eigenvalues; 

(e) two distinct pairs of complex conjugate eigenvalues. 

This lemma shows the principal difference between real and complex spaces because indecomposable 
operators acting in complex spaces have either one or two distinct eigenvalues (Lemma 1 from pP). 


3 Classification of i7-normal Operators Acting in Spaces of Rank 
1 


This section is closely related to 

Let us classify indecomposable 77-normal operators acting in a space 7?" of rank 1. According to Lemma 1, 
we can consider only operators having one of the sets of eigenvalues (a) - (e). However, for a space of rank 
1 not all variants are possible, namely, the alternatives (d) and (e) cannot be realized. Indeed, if 7 V|q,^ 2 (or 
-^^*^Iqi 2 ) two eigenvalues a ± 7/3, the subspace Q 12 is necessarily of dimension 2 or higher. However, 
since Q 12 is neutral, dim Q 12 < 1- Thus, the alternatives (d) and (e) are impossible. Let us consider the 
remaining variants and prove the following theorem: 


Theorem 1 If an indecomposable H-normal operator N (N : 7?" ^ 7?"j acts in a space with indefinite 
scalar product having = 1 negative squares and > 1 positive ones, then 2 < n < 4 and the pair {TV, 77} 
is unitarily similar to one and only one of canonieal pairs Q), (0), 0), 0), 0; 


TV = 


TV = 


N = 


TV = 


Ai 0 
0 A 2 

a (3 

—fj a 

A z 
0 A 


Ai < A 2 , 77 — 772, 


, /3>0, 77 = 772, 


, z = ±l, 77 = 772 


A 1 0 

TV = I 0 A 1 I , 77 = 77fi 
0 0 A 

Air 

TV = I 0 A -1 1 , 77 = 77., 
0 0 A 


( \ 1 Q 0 \ 

0 A 0 cos a 

0 0 A sin a 

\ 0 0 0 A y 


0 0 1 

,0<a<7r, 77=1 0 I 2 0 

1 0 0 


The proof of the theorem is presented in the following subsections. 


( 1 ) 

( 2 ) 

(3) 

(4) 

(5) 

( 6 ) 
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3.1 One Real Eigenvalue of N 

Let us take advantage of Proposition 1 from |2], which is proved for complex spaces but is valid for real ones 
as well: If an indecomposable H-normal operator N : i?" —> i?" (n > 1) has the only eigenvalue then 
there exists a decomposition of i?" into a direct sum of subspaces 


So = {x: {N- \I)x = (7V[*1 - \I)x = 0}, 


(7) 


S, Si such that 


/ N' = XI * * \ / 0 0 I \ 

iV = 0 iVi * , iL = 0 ill 0 , (8) 

V 0 0 N" = \I j \ I 0 0 ) 

where N' : So So, Ni : S —> S, N" : Si —> Si, the internal operator Ni is Hi-normal, and the pair 
{Ni,IIi} is determined up to unitary similarity. To go over from one decomposition i?" = So+S+Si to 
another by a transformation T it is necessary that the matrix T be block triangular with respect to both 
decompositions. 

Since So is neutral, dim So = 1- According to Proposition 2 from [2], if the subspace So is one-dimensional, 
then the operator N is indecomposable. So, it is not necessary to check the indecomposability for each 
canonical form to be obtained in this subsection. As H has one negative eigenvalue, Hi has only positive 
eigenvalues and one can assume that Hi = /, Ni = XI. Later on we will no longer stipulate that Hi = /, 

A^i = XI. By Theorem 1 of |21 (it is also valid for real spaces), n < 4. Consider the cases n = 2, 3, 4 

successively. 


3.1.1 n = 2 

The matrices N and H have form 

"=(( O'"“(? O' 

Since So Si = {0}, a ^ 0. Let vi = ^J\^Vl, V 2 = Then we do not change the matrix H and 

reduce N to form ©■ Since is a special case of canonical form (16) from Theorem 1 (H), the number z 
is an iJ-unitary invariant, i.e., two forms © with different values of z are not H-unitarily similar. 


3.1.2 n = 3 


The matrices N and H have form ©: 


N = 


/ ^ 

a 

b \ 


/ ® 

0 

1 \ 

0 

X 


, H = 

0 

1 

0 

V 0 

0 

A / 


V 1 

0 

0 / 


The condition of the iJ-normality of N is 


2 2 
a = c . 


If a = 0, then c = 0 and V 2 G So, which is impossible because of the condition So H S = {0}. Therefore, 
a ^ 0. Let vi = avi, vo = l/avo. then we reduce N to the form 


/ A 1 6' 

N= 0 A a: 

\ 0 0 A 


a; = ±1 
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without changing the matrix H. If a: = 1, take the i?-unitary transformation T (throughout what follows 
only i?-unitary transformations are used unless otherwise stipulated): 



It reduces N to form 0- If a: = —I, the number h' turns out to be iZ-unitary invariant. Indeed, let 

/01r\ /01r\ 

iV-A/= 0 0 -1 1 iV-A/= 0 0 -1 , 

\000/ \000/ 

and some matrix T = satisfy The conditions 

(9) 
( 10 ) 


NT = TN, 
= L 


Then, according to Proposition 1 from [2], T is block triangular with respect to the decomposition i?" = 
Sq+S+Si, i.e., upper triangular. Condition @ implies 


til — t22 — tss, 

t23 + rt33=ftii-ti2. (11) 

Since the diagonal terms of T are equal to each other, From 1101) it follows that ti 2 + t 23 = 0. Then from 
we get r = f, Q.E.D. The forms obtained are not i7-unitarily similar. Indeed, let an i7-unitary matrix 
T = {tij}ij=i reduce the first form to the second. Since T is upper triangular (Proposition 1 from |2]), from 
@ it follows that tn = t 22 = —^ 33 , which is impossible because condition lUni) implies tutss = 1. Thus, we 
have obtained two canonical forms: 0 and ©■ 


3.1.3 n = 4 


The matrices N and H have form 0: 


/ A 

a 

b 



/ 0 

0 

0 

1\ 

0 

A 

0 

d 

ZJ _ 

0 

1 

0 

0 

0 

0 

A 

e 

5 ^ — 

0 

0 

1 

0 

V 0 

0 

0 

A ^ 


^ 1 

0 

0 

0 / 


The condition of the i7-normality of N is 

a^ + b'^ = d^ + e^. (12) 

Since 0 (otherwise V 2 ,V 3 G Sq, which is impossible), without loss of generality it can be assumed 

that a 0. Taking vi = avi, fq = v^/a, we reduce N to the form 


/A 1 b' c' \ 

0 X 0 d' 

0 0 A e' 

\ 0 0 0 A / 

Further, let us apply the transformation 




/ 


/ VTTF 0 0 0 

0 i/VTTb^ -b'lVTTW 0 

0 67VTTF i/Vi + b'^ 0 

Vo 0 0 i/vrn^ 


5 
















Then we get 


/A 1 0 c" \ 

0 X 0 d" 

0 0 A e" ■ 

\ 0 0 0 A / 

Note that e" ^ 0 because otherwise V3 G S'o, which is impossible because SoH S = {0}. The number e" 
can be replaced by —e" by means of the (i/-unitary) transformation = —V3. So, we can assume e" > 0. 
Moreover, it can be assumed that c" = 0. To this end it is sufficient to take the transformation 


/ 1 0 c'Ve" 

0 1 0 

0 0 1 

\ 0 0 0 


0 

-c"/e" 

1 


\ 


Then c" will vanish, d" and e" will not change. Condition m of the i/-normality of N implies d” = cos a, 
e" = sin a (a € ( 0 ; 7 r)). Show the i/-unitary invariance of the parameter a. Let an iL-unitary matrix 
T = reduce N to the form 


/ A 1 0 0 \ 

0 A 0 cos a 

0 0 A sin d 

\ 0 0 0 A / 


a G (0; tt). 


Then, according to Proposition 1 from [2], T is block triangular with respect to the decomposition i?" = 
S'o+'S’+'S'i and from it follows that ^23 = 0. Now condition oni yields ^32 = 0 . Applying again, we 
have 


til 

^44 COS Of 

t 44 sin a 


^ 22 , 

t22 COS a, 

tss sind. 


Condition (ITnil yields tiit 44 = ^22 = ^33 = 1 so that tu = 1^2 = ^44 = ±1- Hence, cos a = cosd. Since 
sin a, sind > 0, we have ^33 = t 44 and sin a = sind. Consequently, d = a, Q.E.D. Thus, we have obtained 
canonical form ®- 


3.2 Two Distinct Real Eigenvalues of N 

According to Proposition 1, in this case 


N = 


Ai 0 
0 A 2 


f 0 

V a 0 


a 


a 7 ^ 0 . 


It can be assumed that a = 1 (to this end it is sufficient to take vi = vi/a, V 2 = V 2 )- Since the order of 
eigenvalues is not fixed, we can assume that Ai < A 2 . Thus, we have obtained canonical pair O- 


3.3 Two Complex Conjugate Eigenvalues of N 

Let N have two distinct eigenvalues A = a + i/3, X = a — i(3. Since N and A^I*! commute, there exists a 
vector z = x + iy {x,y G i?") such that either Nz = Xz, N'^*^z = Xz or Nz = Xz, N^*'^z = Xz. In the first case 
[z,z] = 0. Indeed, A[z,z] = [Az,z] = [Nz,z] = [z, A^[*lz] = [z,Xz] = X[z,z]. Therefore, (A —A)[z,z] = 0, hence 
[z, z] = 0. Let us write in detail the condition obtained: [x + iy^x — iy] = [a;, x] — i[y, x] — i[x, y] — [y, y] = 0, 
i.e., [x,y] = 0, [x^x] = [y,y]- Since two-dimensional subspace V = span{x,y} cannot be neutral, we 
have [x, x\ 7 ^ 0. Thus, P is a nondegenerate subspace which is invariant for N and . For N to be 
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indecomposable it is necessary to have i?" = V. But [a;,a;] = [y^y], i.e., H is either positive or negative 
definite, which contradicts the condition mm{w_,u+} = 1. Thus, only the case Nz = \z, N^*'^z = Xz is 
possible. It can be shown as before that [z,z] = 0, i.e., [a;, a;] = — [y,y] so that the subspace V = span{x,y} 
is either nondegenerate or neutral. As above, we see that V is necessarily nondegenerate and V = i?". 
Thus, for the basis {x,y} we have 

Let us reduce H to the form £>2 without changing the matrix N. To this end it is sufficient to take 

T — ( 1 

“ ^ -ti2 hi ) ’ 

where 


— 2 <iiti 2 — a, 

''ll ''12 ~ " 


(it can be checked that this system always has a real solution {tii,ti 2 }). Then 



T, 


TN = NT. 


One can replace /3 by —j3 by means of the iL-unitary transformation T = H 2 , therefore, one can assume that 
/3 > 0. Thus, we have obtained canonical pair The proof of Theorem 1 is completed. 


4 Classification of i7-normal Operators Acting in Spaces of Rank 

2 


The objective of this section is to prove the following theorem (the subspace So and the internal operator 
Ni are defined in Section 3.1 by formulas Q, I®, respectively): 


Theorem 2 If an indecomposable H-normal operator N (N : i?" —> acts in a space with indefinite 
scalar product having V- = 2 negative squares and > 2 positive ones, then 4 < n < 8 and the pair {TV, H} 
is unitarily similar to one and only one of the canonical pairs { The list of all the 

canonical pairs is as follows. 

If N has one real eigenvalue X, dim Sq = 1, the internal operator Ni is indecomposable, and n = A, then 
the pair {N,H} is unitarily similar to the canonical pair I 


/ A 

1 

0 

0 \ 

0 

A 

z 

0 

0 

0 

A 

1 

^ 0 

0 

0 

^ / 


H 

= 

Di. 


(13) 

(14) 


If N has one real eigenvalue X, dim Sq = 1, Ni is indecomposable, and n = 5, then the pair {N,H} is 
unitarily similar to one and only one of the canonical pairs { 


( ^ 

1 

0 

0 

0 \ 

0 

A 

1 

0 

0 

0 

0 

A 

1 

0 

0 

0 

0 

A 

1 

V 0 

0 

0 

0 



( 15 ) 
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N = 


(16) 


^ A 1 —n 0 r2 ^ 

0 A 1 ri 0 

0 0 A -1 -n 

0 0 0 A -1 

V 0 0 0 0 A / 

H = D5. (17) 

If N has one real eigenvalue X, dimSo = 1, Ni is decomposable, and n = 4, then the pair {N,H} is unitarily 
similar to one and only one of the canonical pairs {{ UfA) . i2(A} \: 


N = 


/ A 1 0 0 \ 

0 A 0 z 

0 0 A 0 

\ 0 0 0 A / 


z = ±1, 


(18) 


N = 


^ A 1 z 
0 A 0 
0 0 A 
V 0 0 0 


0 \ 

r 

zjr 

A / 


z = ±1, |r| > 1, 


H = Di. 


(19) 

( 20 ) 


If N has one real eigenvalue X, dimSo = 1, Ni is decomposable, and n = 5, then the pair {N,H} is unitarily 
similar to the canonical pair 1 


N = 


/ A 1 0 

0 A 0 
0 0 A 
0 0 0 
\ 0 0 0 


1^2 

2 ' 

Z 

0 

A 

0 


0 \ 
0 


r 


1 

A 


z = ±1, r > 0, 


( 21 ) 


H = Ds- (22) 

If N has one real eigenvalue X, dimSo = Ij *5 decomposable, and n = 6, then the pair {N,H} is unitarily 
similar to one and only one of the canonical pairs 1 \2!A) . { 0,1121)} •• 


N = 


N = 


/ A 
0 
0 
0 
0 
V 0 


1 

A 

0 

0 

0 

0 


0 

1 

A 

0 

0 

0 


0 

0 

1 

A 

0 

0 


0 

0 

0 

0 

A 

0 


0 \ 

- r ^/2 


A 


/ A 

1 

-2ri 

0 

0 


0 


\ 


0 

A 

1 

ri 

0 

-2rl+rll2 



0 

0 

A 

-1 

0 


0 



, > 0 

0 

0 

0 

A 

0 


-1 



0 

0 

0 

0 

A 


r2 




V 0 

0 

0 

0 

0 


X 


) 





/ 0 

0 

0 

1 ^ 





H = 

0 

Do 

0 

0 





0 

0 

1 

0 
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0 

0 

0 ^ 
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(23) 


(24) 


( 25 ) 
















If N has one real eigenvalue X, dim So = 2, and n = 4, then the pair {N,H} is unitarily similar to one and 
only one of the canonical pairs I 


/AO cos a sin a \ 

0 A — sin a cos a 
0 0 A 0 

\ 0 0 0 A y 


0 < Of < TT, 


(26) 


N = 


N = 


/ A 0 0 1 \ 

0 A r 0 
0 0 A 0 

\ 0 0 0 A y 

AO iz z \ 
0 A -z 0 
0 0 A 0 

V 0 0 0 Ay 


|r| > 1, 


z = ±1, 


N = 


f X 0 0 0 \ 

0 A 1 0 

0 0 A 0 

V 0 0 0 A y 


H = 


0 h 
h 0 


(27) 

(28) 

(29) 

(30) 


If N has one real eigenvalue X, dim So = 2, and n = 5, then the pair {N,H} is unitarily similar to one and 
only one of the canonical pairs I I JilMt : 


N = 



/ A 

0 

1 

0 

0 \ 


0 

A 

0 

1 

0 

= 

0 

0 

A 

z 

0 


0 

0 

0 

A 

0 


1 0 

0 

0 

0 

w 

( ^ 

0 

1 

0 

0 \ 


0 

A 

0 

r 

z 


0 

0 

A 

1 

0 


0 

0 

0 

A 

0 


V 0 

0 

0 

0 

A 




/ 

0 

D 

I2 


H = 


0 

h 

0 



V 

I2 

0 

0 


, z = ±1, 


z = ±1, r > 0, 


(31) 


(32) 


(33) 


If N has one real eigenvalue X, dim So = 2, and n = 6, then the pair {N,H} is unitarily similar to one and 
only one of the canonical pairs 1 IHfXP t : 


N = 


/ A 0 1 0 0 0 \ 

0 A 0 1 r 0 

0 0 A 0 1 0 

0 0 0 A 0 1 

0 0 0 0 A 0 

VoooooAy 


r > 0, 


(34) 
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/ A 0 1 0 0 0 \ 

0 A 0 1 r 0 

0 0 A 0 cos a sin a 

0 0 0 A — sin a cos a 

0 0 0 0 A 0 

VOOOO 0 A/ 


0 < a < TT, 


(35) 


/ 0 0 /2 \ 

H= \ 0 h 0 . (36) 

V ^2 0 0 y 

If N has one real eigenvalue X, dim Sq = 2, and n = 7, then the pair {N,H} is unitarily similar to the 
canonical pair { jST)), \HtA) \ : 


N = 


( A 
0 
0 
0 
0 
0 
V 0 


0 

A 

0 

0 

0 

0 

0 


1 

0 

A 

0 

0 

0 

0 


0 0 0 

10 0 

0 0 cos a 

A 0 sin a 

0 A 0 

0 0 A 

0 0 0 


0 \ 

0 

— sin a cos [3 
cos a cos j3 
sin/3 
0 

^ / 


0 < a, /3 < TT, 


/ 0 0 I 2 \ 

i? = 0 /3 0 

\ 12 0 0 y 


(37) 


(38) 


If N has one real eigenvalue A, dim So = 2, and n = 8, then the pair {N, H} is unitarily similar to one and 
only one of the canonical pairs { 


( 


N = 


V 


A 

0 

0 

0 

0 

0 

0 

0 


N = 


A 

0 

0 

0 

0 

0 

0 

0 


0 1 0 0 0 0 0 \ 

A 0 1 0 0 0 0 

0 A 0 0 0 cos a sin/3 sin a sin/3 

0 0 A 0 0 — sin a sin /3 cos a sin fi 

0 0 0 A 0 cos /3 0 

0 0 0 0 A 0 cos j3 

0 0 0 0 0 A 0 

00000 0 A y 

0 <a< 7 r, 0</3< 7 r/ 2 , 

0 1 0 0 0 0 0 \ 

A 0 1 0 0 0 0 

0 A 0 0 0 cos a sin/3 sin a sin 7 

0 0 A 0 0 — sin a sin/3 cos a sin 7 

0 0 0 A 0 cos [3 0 

0 0 0 0 A 0 cos 7 

0 0 0 0 0 A 0 

00000 0 A y 


0 <a< 7 r, 0 < j < (3 < 7 r/ 2 , 

/ 0 0 /2 \ 

H=\ 0 h 0 . 

V ^2 0 0 y 


(39) 


(40) 

(41) 
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If N has 2 distinct real eigenvalues Ai, A 2 , then the pair {iV, iJ} is unitarily similar to the canonical pair 


N = 


4 Ai 

1 

0 

^ \ 

0 

Ai 

0 

0 

0 

0 

A 2 

0 

^ 0 

0 

T 

A 2 ) 


for r ^ 0 Ai < A 2 , 


H = 


0 

h 


h 

0 


If N has 3 eigenvalues: X G R, a ± i/3 /a,/3 G /fi, l3 > 0), then the pair {N,H} is unitarily 
canonical pair 

/a /3 0 0 \ 


N = 


-P a 0 0 

0 0 A 0 

\ 0 0 0 A y 

0 I 2 
h 0 


H = 


If N has 4 eigenvalues: oti ± i/3i, 02 ± iP 2 , (oii^ Pi,o: 2 , P 2 G 0 < Pi < P 2 , 0.1 < a 2 if Pi 
pair {/V, iJ} is unitarily similar to the canonical pair { i4h}) , 141)) } : 


N = 


I Oil Pi 
—Pi ai 
0 0 

0 0 




0 0 

0 0 

a2 ZP 2 

-ZP 2 a2 j 


z = ± 1 , 


H = 


0 

I 2 


h 

0 


If N has 2 eigenvalues a±ip (a, P G P > 0), and n = 4, then the pair {N, H} is unitarily 
and only one of the canonical pairs { 


N = 


( a P cos 7 sin 7 \ 

—/3 a. —sin 7 cos 7 

0 0a P 

V 0 0 -/3 a 


0 < 7 < 27r, 


/V = 


f a P 0 
—P a 1 
0 0a 
\ 0 0 p 

0 I 2 

I2 0 


1 \ 
0 

-p 

a J 


H = 


If N has 2 eigenvalues a2zip (a, P Gift, P > Q), and n = 6, then the pair {N, H} is unitarily 
and only one of the canonical pairs {El,El}, {E1,I21}- 


(42) 

(43) 

similar to the 

(44) 

(45) 

= P 2 ), then the 

(46) 

(47) 

similar to one 

(48) 

(49) 

(50) 

similar to one 


N = 


( a P 
—P a 
0 0 
0 0 
0 0 
V 0 0 


0 0 0 

0 1 (cos 7 + l)/4/3 — r 

a P i(cos 7 + l) 

—/3 a — isin 7 
0 0 a 

0 0 -p 


r 

sin 7/4/3 
\ sin 7 
i(cos7- 1 ) 

P 

a 


\ 
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(51) 



VI 

o 

7 

A 

to 

-.2 





/ 

a 

P 

0 

0 


r 

0 

\ 


-p 

a 

0 

1 


0 

r 



0 

0 

a 

P 


0 

0 



0 

0 

-P 

a 


0 

-I 



0 

0 

0 

0 


a 

P 


V 

0 

0 

0 

0 


-P 

a 

/ 



/ 

0 

0 

I 2 

\ 




H = 


0 

I 2 

0 

• 





V 

h 

0 

0 





(52) 


(53) 


If N has 2 eigenvalues azLi/d (a,/3 G 3?, /3 > 0), and n = 8, then the pair {N,H} is unitarily similar to the 
canonical pair 


/ 

N = 

V 


a P 
—/3 a 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 


0 0 
0 1 
a P 
—P a 
0 0 
0 0 
0 0 
0 0 


0 0 
0 0 
0 0 
0 0 
a P 
—P a 
0 0 
0 0 


0 

sin^ 7/2/3 

. 2 

Sin 7 

— sin 7 cos 7 cos 5 
sin 7 cos 7 sin 5 
0 


a 

-P 


0 \ 

sin 7 cos 7 cos <3/2/3 
sin 7 cos 7 cos <5 
— cos^ 7 
0 

sin 7 cos 7 sin 5 

P 

a ) 


0 < 7 < 

7 r/ 2 , 

0 

A 

A 

(54) 

/ 

0 

0 

I 2 \ 



0 

h 

0 . 

(55) 

V 

h 

0 

0 / 



Here all parameters are H-unitary invariants, i.e., the same canonical forms are H-unitarily similar to each 
other iff the values of all parameters are equal. 


The proof of the theorem is presented in what follows. 


4.1 One Real Eigenvalue of N 

The case when N has only one real eigenvalue A can be considered as in |5]. Namely, if dim Sq = 1, then 
there exists two alternatives: Ni is indecomposable or decomposable, this property being independent of 
the choice of the decomposition ii" = S'o+S'-i-S'i because the indecomposability or decomposability of Ni 
does not change under unitary similarity of the pair {A^i, Hi}. In the former case one can show that n < 5 
and obtain the canonical forms in the latter one can show that n < 6 and obtain 

the canonical forms {CHI),®} -1®,®} in just the same way as it was done in [5]. If the subspace So 
is two-dimensional, the operator N can also be considered as in d except for the case n = 4 because one 
of the corresponding canonical forms in d is essentially complex. Thus, for the case when N has one real 
eigenvalue A we will consider only the alternative dim Sq = 2, n = 4 and omit the rest. 

4.1.1 dim So = 2, n = 4 
In this case = 5'o-i-S'i. Therefore, 

/ 0 
0 
0 

V 0 


N -XI = 


0 N2 
0 0 


0 a b \ 

0 C d rr^f 0 l2\ 

000 ’ V/2 0y’ 

0 0 0 / 
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and the submatrix N 2 is not restricted by the condition of the if-normality of N. 
(a) detN 2 ^ 0. Suppose an i?-unitary transformation 


reduces N — XI to the form N — XI: 


T = 


Ti T2 \ 
T3 T4 J 


N -XI = 


( n 


, TV - A/ = ( 

0 

N2 \ 

V 0 

0 y 


V 0 

0 y 


Then conditions (I5till - 15811 below are necessarily satisfied: 


N 2 T 3 = 0, (56) 

N 2 T 4 = T 3 N 2 , (57) 

0 = T 3 N 2 . (58) 

Since N 2 is nondegenerate, (innii is satisfied only if T 3 = 0. The operator T is i7-unitary iff 

T 3 TI = J, (59) 

TiT; + T 2 T^ = 0 . (60) 


It follows from system - iinnii that without loss of generality we can consider only quasidiagonal trans¬ 
formations T = Ti 0 because T 2 does not appear in equations llCTi - 
Thus, the only condition 

N 2 = T 1 N 2 TI (61) 

should be satisfied, i.e., it is necessary to find out what form a nondegenerate 2 x 2 -matrix N 2 can be reduced 
to under congruence. 

Consider the matrix N 2 = N 2 N 2 . Its spectral characteristics are invariant because N 2 = T 1 N 2 ■ 
Since det = 1, N 2 has either two complex conjugate eigenvalues cos a ± zsino or two real eigenvalues r, 
1/r (r ^ 0). In the former case N 2 can be reduced to the form 




COS a sin a 
— sin a cos a 


0 < a < TT, 


(62) 


in the latter to the Jordan normal form. 
If N 2 has form I62II . then 


N 2 = 


t sina/(l — cos a) t 

—t t sina/(l — cosa) 


t 0 . 


As det N 2 = 2 t^/{l — cos a) > 0, one can take Ti = det N 2 I and obtain 


N. = 


± cos J ± sin Y 


0 < a < TT. 


0 sin ^ ± cos J 

Since the transformation Ti = D 2 replaces sin ^ by — sin ^, we can write 


JV 2 


cos a sin a 
— sin a cos a 


0 < a < TT 


(63) 


(note that two last formulas for N 2 are not equivalent because 16311 includes the extra value a = 7^12 
corresponding to the case 77^ = ~^)- 
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Now we must prove the invariance of the parameter a. To this end suppose that a nondegenerate matrix 
Ti satisfies (int . where N 2 has form (ES and 


No = 


cos a sin a 
— sin a cos a 


, 0 < a < TT. 


As N2 + = Ti{N2 + N2 )T^ and N2 - = Ti{N2 - N2 )T^, we have 

4 cos^ a = det{N2 + N2) = (detTi)^ det{N2 + N^) = [detTifAcos^ a 

and _ 

4 sin^ a = det{N2 — N^) = (detTi)^ det{N2 — = (detTi)^ 4 sin^ a. 

Therefore, \detTi\ = 1, cosa = ± cosa, sina = sind. Now we write the condition A^2+A^2 = Ti(A^2+A^2 )Ti 
in detail: 


cosa 0 
0 cosa 


(til + ^ 12 ) cos d (tiit 2 i + ^ 12 ^ 22 ) cos ( 

(^ 11^21 + ^ 12 ^ 22 ) cos d (^21 + ^ 12 ) cos d 


Since | cosa| = | cosd|, we have + ^12 = hence cosa = cosd. Thus, a = d, Q.E.D. 

If N 2 has distinct real eigenvalues r and l/r, i.e., r ^ ±1, then it can be reduced to the diagonal form 
N 2 = l/r0r, |r| > 1. Consequently, 


N2 = 

Taking Ti = 1 0 t, we reduce N 2 to the form 

N2 = 


0 t 
rt 0 


0 1 
r 0 


t ^ 0. 


, |r| > 1. 


(64) 


It is clear that r is an invariant. 

Finally, we consider the case when N 2 has the eigenvalues ±1. If N 2 = I, the matrix N 2 is selfadjoint, 
hence, it can be reduced to the diagonal form. Therefore, the nondegenerate subspace V = span{vi,V 3 } is 
invariant both for N and for N^*\ i.e., the operator N is decomposable. It can easily be checked that N 2 is 
not equivalent to the form 




1 I 
0 I 


because then N 2 turns out to be degenerate, which is impossible, 
above-mentioned form 


N2 


0 

-1 


I 

0 


If N 2 = —I, N 2 can be reduced to the 


The last case to be considered is the case when the Jordan normal form of N 2 is 


Then 


Taking Ti = y^/, we achieve 



(65) 
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Here z is an invariant. Indeed, suppose that some matrix Ti satisfies condition where 



Then = ^thz, hence z = z. 

As a result, we have obtained three forms (161111 . (I64II . (I65II . Now it is necessary to find out whether the 
operator N is indecomposable in the three cases. The indecomposability of N means that {aN 2 + 6 A^|)a; = 0 
only if (cc, N 2 X) = 0 (a^ + 7 ^ 0). If N 2 = A^ 2 A^ 2 ~^ no real eigenvalues, the equation {aN 2 + bN^)x = 0 

has no solutions, i.e., N is indecomposable if N 2 has form with a ^ Ti 12. If an eigenvalue A of N 2 is not 
equal to 1, then (x, N 2 X) = 0 because (x, N 2 X) = (x, XN^x) = A(x, N^x) = A(x, N 2 X). Thus, if N 2 has form 
(1?^ . (EU, or (li)^ with a = 7 r/ 2 , then N is also indecomposable. 

(b) detN 2 = 0. Since N with A ^2 = 0 is decomposable, it suffices to consider the remaining case rgN 2 = 1: 


It is readily seen that Sq IH 7 ^ {0} if la = kb, therefore, we can assume that this condition is not satisfied. 
Taking T = 0 where 


Ti = 


a k \ 
b I ) 


we obtain one more canonical form: 


/ 0 

0 

0 

0 \ 

0 

0 

1 

0 

0 

0 

0 

0 

1 0 

0 

0 

0 y 


(it can easily be checked that this form is indecomposable). 

As a result, we have proved that 

If an indecomposable H-normal operator N (N : —> C^) has the only eigenvalue A G 3?, and 

dim So = 2, then the pair {N, H} is unitarily similar to one and only one of the canonical pairs { tPM) . . 


4.2 Two Real Distinct Eigenvalues of N 

Since the canonical pair {63,63} is obtained in the same way as in | 3 , we will not repeat the proof of the 
following fact: 

If an indecomposable H-normal operator acts in a space i?" of rank 2 and has 2 distinct real eigenvalues: 
Ai and A 2 , then n = 4 and the pair {N,H} is unitarily similar to the canonical pair { i4^ > ifd}) } . 


4.3 Three Eigenvalues of N: One Real and Two Complex Conjugate 

Suppose an indecomposable tt-normal operator N has a real eigenvalue A and two complex eigenvalues 
a± if3 {P > 0). According to Lemma 2.1, we have i?” = Qi-i-Q 2 , dim Qi = dim Q 2 = m, [Qi, Q\] = 0, 
[Q 2 , Q 2 ] = 0, A^Qi C Qi, NQ 2 C Q 2 , Ni = A^Iqj has two eigenvalues a ± i/3, N 2 = A^Iqj one eigenvalue A. 
Since TOzn{x_,x+} = 2, n = 0 > 4. On the other hand, the subspaces Qi and Q 2 are neutral so that 

n = 2m < 4. Thus, n = 4. As iL is nondegenerate, for any basis in Qi there exists a basis in Q 2 such that 


H = 


0 I 
I 0 


Take a basis in Qi such that 


Ni 


a /3 A 
-/3 a J ' 


( 66 ) 
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(67) 


Then with respect to the decomposition i?" = Q 1 + Q 2 we have 


N = 


Ni 0 
0 N 2 


, H = 


0 I 
I 0 


The condition of the iJ-normality of N is 

Nin; = n;ni. 

The only matrix commuting with ll?^ and having one eigenvalue A is XI. Thus, 


( 68 ) 


"=( “,3 f)®(o A 

It can easily be checked that N is indecomposable. Indeed, suppose a subspace V is invariant for N and 
. Since min{dim V, dim } < 2, we can assume that dim V < 2. U V were of dimension 1, then there 
would exist a vector v G V such that Nv = Xv, N^*^v = Xv. But all eigenvectors of N corresponding to the 
eigenvalue A are not eigenvectors of Thus, dim V ^ 1. Suppose dim V = 2. Then N\v has either the 
only eigenvalue A or two eigenvalues a ± i/3. In the former case V — Q 2 , in the latter V — Qi. In the both 
cases V is degenerate, therefore, N is indecomposable. 

Thus, we have proved that 

If an indecomposable H-normal operator acts in a space i?" of rank 2 and has 3 eigenvalues: X G R, a±z/3 
(a, /3 G 3?, [3 > 0), then n = 4 and the pair {N, H} is unitarily similar to the canonical pair I i44{ l: nW }- 


4.4 Two Distinct Pairs of Complex Conjugate Eigenvalues of N 


Suppose N has four eigevalues ai ± ifh, 02 ± iP 2 (/3i, 132 > 0, (ai,/3i) ^ ( 02 ,/32))- Let us fix the order of 
these pairs: /3i < P 2 , ai < 02 if Pi = P 2 . As in the previous section, one can show that N and H can be 
reduced to form O) with 


Ni 


ai Pi \ 

—Pi ai J 


It follows from condition P|l of the i7-normality of N that 


N 2 


02 ZP 2 
— ZP 2 02 


Z = ±1. 


Now we prove that the number z is an iJ-unitary invariant. To this end suppose that a matrix T satisfies 
condition @ NT = TN and condition (IIOH TT^*! = I, where 


" = t) - " 

It follows from ® that T = 0 T 2 , where 



It follows from UlOll that T 2 = Tf therefore. 



= A^i © 


til ti2 
— tl2 til 


^33 ^34 

— ^34 ^33 


02 

-ZP 2 


ZP 2 

O 2 


|z| = |5| = I. 


It is seen that under these conditions z = z. The indecomposability of the form obtained can be checked as 
before. 

Thus, we have proved that 

If an indecomposable H-normal operator acts in a space i?" of rank 2 and has 4 eigenvalues: oi ± iPi, 
02 ± iP 2 , (oi, /3i, 02 , P 2 G^, Q < Pi < P 2 , oi < 02 if Pi = P 2 ), then n = 4 and the pair {N, H} is unitarily 
similar to the canonical pair { i4h\ l, i4'l] l} ■ 
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4.5 Two Complex Conjugate Eigenvalues of N 

The two following propositions hold for any space with indefinite scalar product. They are in a sense 
analogous to Propositions 1, 2 from |2]. 

Proposition 2 Let an indecomposable H-normal operator N acting in i?” (n> 2) have two distinct eigen¬ 
values A = a + i(3, \ = a — i/d. Let 

S'o = = X -\- iy {x,y G i?”) : Nz = Xz, N^*^z = Xz}, 

Sq = {z = X -\- iy (x, y G i?") : Nz = Xz, N^*^z = Xz}, 

{^j}i ({^j}p+i) ^6 ® basis of Sq (Sq), and 


p+q 

Sq = y^^span{xj,yj}. 

j=i 


Then there exists a decomposition of R"' into a direct sum of subspaces Sq, S, Si such that 


N = 


/ N’ 


* ) 


/ ^ 

0 

I \ 

0 

Ni 

* 

1 ’ ^ = 

0 

Hi 

0 

V 0 

0 

N" ) 


V / 

0 

0 / 


(69) 


where 


N' : 

Sq- 

-^Sq, 

N' 

= N[®.. 

• ® K+q^ 


N'- 

j 


a 

-p 


, j = 1 ,■ 

..p + q. 

(70) 

N" : 

Si - 

^Si, 

N" 

© 

II 

..©a^;+9^ 



N'; = N} ifl<j<p, N” = N'* ifp<j<p + q, (71) 

the internal operator Ni is Hi-normal and the pair {Ni,Hi} is determined up to unitarily similarity. To 
go over from one decomposition i?" = 5'o+>5'+>S'i to another by means of a transformation T it is necessary 
that the matrix T be block triangular with respect to both decompositions. 


Proof: It is clear that the subspace Sq is well defined, i.e., that its definition does not depend on the choice 
of bases in Sq and Sq. Since N and commute and have two eigenvalues, at least one of the subspaces 
Sq, Sq is nontrivial so that p-\- q > 0. Show that the system {xj}^‘‘ U {yj}^'^ is a basis in Sq. In fact, the 
assumption = 0 (o^, bj G 3?, _) = 1,.. .p-\-q) means that 'R-oJ2^^iiO'j~ibj)zj = 0 , therefore, 

-ibj)zj} = 0. But ne{N -ibj)zj} = aTZeJ^P.t^^iaj -ibj)zj - -ibj)zj 

so that iTmJ2^=i{0'j = 0- Thus, — 0- Since the vectors Zj are linearily independent 

in C”, Oj = bj = 0 {j = 1,.. .p -\- q), i.e., the vectors {xj}^‘^ U {yj}^‘^ are linearily independent in i?”. 
Thus, the dimension of Sq is equal to 2{p-\- q). 

Now let us prove that for N to be indecomposable it is necesssary that Sq be neutral. Indeed, we 
already know that z = x -\- iy {x,y G i?") is an eigenvector of such that Nz = Xz, then the 
subspace span{x,y}, which is invariant for N and N^*\ is either nondegenerate or neutral (see Section 2.3). 
Since n > 2 and N is indecomposable, it is necessarily neutral. Further, if Nzi = Xzi, N^*''zi = Azi, 
Nz 2 = Xz 2 , N'^*^Z 2 = Xz 2 , then it can be shown (as in Section 2.3) that [ 21 , 22 ] = [ 21 ,^] = 0, hence 
[a;i,a; 2 ] = [a;i,y 2 ] = [yi,a; 2 ] = [yi,y 2 ] = 0. If Nzi = Xzi, N^*hi = Xzi, Nz 2 = Xz 2 , N'^*h 2 = Xz 2 , then 
[ 21 , 2 ^] = 0, i.e., [a:i,a; 2 ] = [yi,y 2 ] and [xi,y 2 ] = -[yi,a; 2 ]- If 7 ^ 0 (a = [a:i,a; 2 ], b = [xi,y 2 ]), the 

two-dimensional subspace span{axi — byi + X 2 , bxi -\- ayi + 1 / 2 }, which is invariant for N and N^*\ will be 
nondegenerate, therefore, N will be decomposable. Thus, for N to be indecomposable it is necessary to have 
a = b = 0. It can be checked in the similar way that the conditions [xi, X 2 ] = [ 2 / 1 , 2 / 2 ] = [xi, 2 / 2 ] = [ 2/1 > 2 : 2 ] = 0 
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are satisfied if Nzi = Azi, N^*^zi = Xzi, Nz 2 = Xz 2 , N^*^Z 2 = Xz 2 - Thus, if N is indecomposable, 5*0 is 
neutral. 

For any neutral subspace Sq of a space with indefinite scalar product there exists a subspace such that 

^l(So+Si) = 



Since (S'o+S'i) is nondegenerate, the subspace S = is nondegenerate too and i?” = Sq+S+Si. 

It is clear that with respect to this decomposition the matrices N and H have form (P|l . the submatrix 
N' has form GDI and N" has from GD- The last two statements of the proposition can be proved as in 
Proposition 1 from The proof is completed. 


Proposition 3 An H-normal operator such that dim Sg = 2 is indecomposable. 

Proof: Assume the converse. Suppose some nondegenerate subspace V is invariant both for N and for 
Let us denote Vi = V, V 2 = Ni = N\vi, N 2 = N\v 2 , Hi = H\vi, H 2 = H\v 2 - Since the operators Ni 
{i = 1,2) are iLi-normal, both subspaces Sq'^ C Vi (defined as So) are nontrivial, i.e., dim Sq'^ > 2. Since 
So = Sq^^+sI^\ dim Sq = dim Sq^'^ + dim S^^ > 4. This contradicts the condition dim So = 2. Thus, N is 
indecomposable. 

Now let us show that if min{v-,v+} = 2, then N is indecomposable only if n < 8 . According to 
Proposition 2, which is applicable (recall that n = > 4), if A/ is indecomposable, then Sq is neutral 

so that dim So = 2. Therefore, if we show that for n > 8 we have dim So > 2, this will mean that N is 
decomposable. 

Let us complexify the source space i?" and apply the results from ^ and |2] concerning the decomposition 
of an iL-normal operator in a complex space. Lemma 1 from ^ states that for an iL-normal operator having 
two distinct eigenvalues A and A there exists a decomposition of C" into a sum C" = Vi-i- 1 ^ 2 + 14 +14 such 
that 


/ 

Ni 

0 

0 



( ^ 

I 

0 



0 

N 2 

0 

0 


I 

0 

0 

0 


0 

0 

A4 

0 

0 

0 

Ho 

0 

V 

0 

0 

0 

Ni yi 


1 0 

0 

0 

Hi y 


where Ni, N 3 have the only eigenvalue A, N 2 , N 4 the only eigenvalue A, dimVi = dimV 2 . It is seen that if 
the space C” is i?” complexified, then dimVo = dimVji. 

Since ranks of the subspaces Vi-i-P 2 , V 3 , 14 are less than or equal to 2, Theorem 1 from P and Theorem 1 
from 121 are applicable. It follows from these theorems that if dim 14, dim V 3 > 0, then there exist at least 
two linearily independent vectors zi, Z 2 such that Nzi = Xzi, N'^*^zi = Azi, Nz 2 = Xz 2 , N^*^Z 2 = Xz 2 , 
i.e., dimSo > 4. If dimVo = 0, n is equal to 4 because the subspaces 14 and 14 are neutral (hence 
n = (2 dimVi) < 4 => n = 4). If dimVi = 0, there appear two alternatives: 14 and 14 each have rank 1 
or one of these subspaces has rank 0. In the latter case either N 3 or A4 is decomposable for any n. In the 
former case, according to Theorem I P, A^a (^^ 4 ) is always decomposable if dimVs > 4 {dimVi >4). In 
either case for n > 8 there exist two linearily independent vectors zi, Z 2 such that Nzi = Xzi, = Azi, 

Nz 2 = Xz 2 , N^*^Z 2 = Xz 2 . As above, we have dim So > 4. Thus, if n > 8 , Ai is decomposable, Q.E.D. 

Thus, according to Proposition 2, the matrices N and H can be reduced to the form 


where 


N = 


/ Ni 

N 2 

No \ 


0 

I \ 


0 

Ni 

No 

, iL = 0 

I 

0 , 

(72) 

V 0 

0 

No / 


0 

0 / 



Ni 


a P \ 
-pa)' 
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Nq is equal either to or to N*. The condition of the _ff-normality of N is equivalent to the system 


iViiV* = 7V*fVi, (73) 

NiNI + N 2 N^ = n;n 2 + n;n4, (74) 

n^n; + N2N* + N3N* = n;n3 + n;n 5 + N^Ne, (75) 

= N^Ni. (76) 


Note that if Nq = N*, then dim Sq > 0 so that it is the case dimVi > 0. It was stated before that if 
dimVi > 0, then for indecomposable operators n = 4. Therefore, for n = 4 the submatrix Nq can be equal 
to either Ni or N* but for n = 6 , 8 we have Nq = Ni. Now let us consider the cases n = 4, 6 , 8 successively. 


4.5.1 n = 4 

By the above, 


N = 


f Ni 

Ns \ 

V 0 

Ne J 


( 

a 

P 

a 



-f3 

a 

c 

d 


0 

0 

a 

±(3 

\ 

0 

0 

tP 

a y 


Nq = Ni Then from (17?^ it follows that c = —b, d = a. If a^ + 6 ^ = 0, i.e., N 3 = 0, then Sons'! 7 ^ 0, which 
contradicts the indecomposability of N. Therefore, + b^ ^ 0. Taking the block diagonal transformation 
T = + 62/2 0 1/^02+62/2, we can reduce N to the form 


N = 


( a (3 cos 7 sin 7 \ 

—/3 a —sin 7 cos 7 

0 0a P 

\ 0 0 -/3 a J 


0 < 7 < 27 r. 


(77) 


According to Proposition 3, matrix (EH) is indecomposable. Let us prove the //-unitary invariance of the 
parameter 7. To this end suppose that a matrix T satisfies conditions 


NT = TN, 

_ j 


(78) 

(79) 


for the matrix N of form CZZI) and the matrix 


N = 


According to Proposition 2, the matrix T has the block triangular form 




/ 

a 

/3 

COS 7 

sin 7 

\ 

( Ni 

Ns \ 


-/? 

a 

— sin 7 

cos 7 


V 0 

Ni ) 


0 

0 

a 

a 




1 

0 

0 

-f3 

a 

) 


0 < 7 < 2tt. 


T = 


Ti T 2 

0 T 3 


with respect to the decomposition = So-i-Si. The transformation T is //-unitary iff 


Tit; = /, 

TiT^ + T 2 T/ = 0. 

It follows from condition EHl) that A^i and Ti commute, therefore. 


Ti 


til ti2 \ 

— tl2 til J 


(80) 

(81) 
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so that from m we get 


tl3 ti4 
-tli tl3 


T2 = 


Now, combining 18011 and I78II . we have N 1 T 2 + ^ = T 1 N 3 + T 2 N 1 . But T 2 and 7Vi commute (as well as 

Ti and N 3 ) so that N 3 = T 1 N 3 T* = N^TiT^ = (detTiYN^. Since detN^ = detN^ = 1, we have (detTi)^ = 1 
and N 3 = N 3 , i.e., 7 = 7, Q.E.D. 

Nq = Then, according to G3, c = b. The transformation 


reduces N 3 to the form 


/ 1 0 0 a/2/3 \ 

0 1 -a/2/3 0 

0 0 1 0 

\ 0 0 0 1 / 



without changing the submatrices Ni and Ng. If both b' and d' are equal to zero, the condition S'o H iSi = {0} 
fails. Therefore, 46'^ + ^ 0 and we can take the transformation 


^ coscj) sint/) —rsint/) rcoscj) \ 

— sin^ cos(/> —r coscj) —rsincj) 

0 0 cos (j) sin (j) 

y 0 0 — sin (/) cos cj) J 


where cos2(/ = 2b'/, sin2^ = —d'/^/d''^ + 46'^, r = d'/(4/3). It does not change Ni and Nq but 
reduces N 3 to the form 

= ( b" o' ) ’ ^" = + > 0. 

If we now take vi = Vt^vi, V 2 = \fWv 2 -, V 3 = wa/v^, V 4 = v^j^/W, then /V3 will be equal to 02 - Thus, we 
have obtained the final form for the matrix N\ 


N = 


fa /3 0 1 \ 

-/3 a 1 0 

0 0 a -/3 

V 0 0 /3 a 


(82) 


According to Proposition 3, matrix (|H3) is indecomposable. Forms (f77| and (|H3) are not iJ-unitarily 
similar because for matrix the subspace S'f defined in Proposition 2 is nontrivial in contrast to that for 
G3- Thus, we have proved that 

If an indecomposable H-normal operator acts in a space of rank 2 and has 2 eigenvalues: a i(3 
(a, 3 G (3 > 0), then the pair \N,H} is unitarily similar to one and only one of the canonical pairs 

mm}- 


4 . 5.2 n = 6 


The matrices N and H have form Ga) with iVg = Ni. Since the submatrix N 4 is an ordinary normal matrix 
(condition itTHll b one can assume that A^4 = A^i. Thus, 


N2 N3 
Nx N5 
0 Ni 


N = 


iVi 

0 

0 
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First reduce the submatrix 


to the form 


N 2 = 


a 

c 


b 

d 



without changing the submatrices A^i = -/V 4 = Nq. To this end take 


/ I T 2 -^T 2 T* \ 

r = 0 / -T* , 

Vo 0 I J 


(83) 


(84) 


f b /(3 -a/p \ 

Vo 0 )■ 


Then 


N 2 


0 0 \ 

c' d' )■ 


If both c' and d! are equal to zero, i.e., A^2 = 0 , then from condition of the i/-normality it follows that 
A^5 = 0 , which contradicts the condition So r\ S = { 0 }. Therefore, c'^ + ^ 0 and we can subject the 

matrix N obtained to the transformation T = /2 0 Ti © /2, where 


( dl/^d'^ 0 d'2 c7Vc'2 0 d'2 
V -cVVc'2 0 d'2 d7Vc'2 0 d'2 ) ■ 


Then 

= ( 0 (?' ) ’ > 0 - 

Taking v/ = d"vi, V 2 = d"v 2 , V 3 = V 3 , v/ = V 4 , €5 = vo/d'/ vg = vg/d", we obtain desired form 18311 for the 
submatrix N2. 

Now let us apply conditions m and We get 


Ng = 


1 

2 


cos 7 0 1 sin 7 
— sin 7 cos 7 — 1 


0 < 7 < Stt, 


Ng = 


p q 

(cos 7 0 l)/4/3 — g sin7/4/3 0 p 


Finally, take transformation ilMli with 

T2 = 2p/ (cos 7 0 1) I2 if 7 7 TT, 


T 2 = 


0 -q 
q 0 


if 7 = TT. 


Then 


Ng = 


0 q 

(cos 7 0 l)/4/3 — 9 ' sin7/4/3 

Ng = p'h (7 = tt). 

As a result, we have obtained two forms: 

( a P Q Q 0 

—P a 0 1 (cos 7 0 1)/4/3 — r 

Q Q a P i(cos7 0 l) 

0 0 —P a — isin7 

0 0 0 0 a 

V 0 0 0 0 -p 


N = 


(7 7 tt), 


r 

sin 7/4/3 
^ sin 7 
i(cos7- 1) 

P 

a 
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( 85 ) 


N = 


( 86 ) 


0 < 7 < 27r, 7 ^ TT, 

/ a P 0 0 r 0\ 

—/3 a 0 1 0 r 

0 0 a /3 0 0 

0 0 -P a 0 -1 

0 0 0 0 a P 

VO 0 0 0 -/3 a/ 

According to Proposition 3 , matrices (ISSl) and ijsnii are indecomposable. Let us show that they are not 
iL-unitarily similar and that the numbers r and 7 are iL-unitary invariants. To this end suppose that some 
iL-unitary matrix T reduces the matrix N to the form N: 


/ Ni 

N2 

Ns \ 

/ Ni 

N2 

Ns \ 

N= 0 

Ni 

Ns 

, 0 

Ni 

Ns 

V 0 

0 

Ni J 

V 0 

0 

Ni ) 


where 


N, = 


a P 
—P a 


Nn = 


N5 = ^ 
% = 


smy 
cos 7 — 1 


cos 7 + 1 

— sin 7 

cos 7 + 1 sin 7 

— sin 7 cos 7 — 1 


0 0 
0 1 


0 < 7 < 27 r, 


0 < 7 < 27 r. 


Then, according to Proposition 2 , T has the block triangular form 


T = 


Ti T2 Tg 

0 Ti Ts 

0 0 Tg 


with respect to the decomposition Rp = S'o-i- 5 '-i-S'i. It follows from condition itTHIl NT = TN that 


Ti = T4 = Tg = 


cos (j) sin (p 
— sin d> cos d> 


To = 




tii 


—tl4 tl3 + 


sin (j) 


Condition 17911 TT^*! = I implies T1T5* + T2T4 = 0 , hence 


Tg = -TiT;Ti = 


^35 


^36 


^36 ^35 Q 


where 


tgg = —ti3 cos2^ — ti4 sin20 + 
tgg = —ti3 sin 2(/) + ti4 cos 2(/i— 


sm 


P ’ 

cos (p sin^ p 

P ■ 


Substituting the expres^ns for T4, Tg, Tg in the formula N1T5 + A^gTg = T4N5 + T5N1, which follows from 
m, we obtain: = N5. Therefore, forms IHSl) and are not iJ-unitarily similar and the parameter 7 

is an iJ-unitary invariant. 

Now let us check the 7 J-unitary invariance of r for matrix ilSBli . To this end suppose that 


Ns = 


0 r 

(cosy + l)/ 4/3 — r sin7/4/3 
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N. = 


0 f 

(cos7 + l)/4/3 — f sin7/4/3 


0 < 7 < 27r, 7 7 ^ TT. It follows from that TiTg = — ^T 2 T 2 * + X, where X is an antisymmetric matrix, 


therefore. 


where 


Ts = 


^15 ^16 

^25 ^26 


a;sm0 —xcosfp 
X cos (j) X sin (p 


2ti5 = -(ti3+ti4)cos(/) + ti4sin^(^//3, 

2ti6 = — (ti3 + ti4) sin;/) — ti4 sin^cos^// 3 , 

2^25 = -ti4sin(^cos(/)//3+((ti3+sin(/)// 3 )^+ti4)sin(^, 

2^26 = -ti4sin^(/)//3-((ti3+sin(^//3)^+t44)cos(;i. 

Since NiT^ + N2T^ + N^Tq = TiN^, + T2+ T^Ni (condition (I 78 ll i. = T4* [NiT^ — T^Ni + N2T^ — T2-/V5 + 
N^Tq). Substituting the expressions for T2, Ts, T5, Tg in this formula, we obtain: 

01^13 + 02^14 + 03 = 0 , ( 87 ) 

biti3 + ^2^14 + 63 = f -r, ( 88 ) 

where 


«! = --(C0S((() - 7) + COS(()), 

02 = -i(sin( 0 - 7 )+ sin 0 ), 

03 = -^sin(()(cos((;!)- 7 )+ cos(()), 

61 = i(sin((()- 7 )-sin(/)), 

62 = -i(cos((()- 7 )-cos(()), 

63 = ^ sin(^(sin((^ - 7) - sini^). 

Since the left hand sides of equations 1HZ|) - (IHHli are proportional and the coefficients of ti 3 and of +4 in (113 
are not equal to zero simultaneously, condition JHTIi implies f = r. Therefore, r is an i7-unitary invariant. 
The proof of the invariance of r for matrix is analogous. 

Thus, we have proved that 

If an indecomposable H-normal operator acts in a space i?® of rank 2 and has 2 eigenvalues: a iP 
fa, 3 G Sff, 3 > 0), then the pair \N,H} is unitarily similar to one and only one of the canonical pairs 

imw- 

4 . 5.3 n = 8 

The matrices N and H have form GUi, Nq being equal to Ni: 

Ni N2 N3 
fV= ( 0 N 4 N 5 

0 0 Ni 


Since N 4 is an ordinary normal matrix (condition (17611 1. it can be assumed that Afr = iVi 0 A^i. 
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Having these equalities in mind, we reduce the submatrix 




a b c d \ 
e f 9 h ) 


to the form 


N 2 


0 0 0 0 \ 
0 1 0 0 j 


(89) 


without changing the submatrices -/V4, and Nq = -/Vi. To this end take transformation with 


( b/P -a/p d/p -c/P \ 

Vo 0 0 0 y ■ 


Then 


N 2 


0 0 0 0 \ 

e' /' 9 ' h' )■ 


Now subject the obtained matrix N to the transformation T = I 2 (B T/ (B T” 0 I 2 , where 


n 


f/V e’^ + e'/Ve^TT^ 

-e'/v'e'^ 0 /'2 /7^e'2 0 /'2 


if e'2 0 /'2 > 0 , 


T/ 


T[ = I 2 if e' = /' = 0, 

h’/y/ g'^ + h'^ g'/y /g'^ 0 h’^ \ 

-g'/yfg^^^Th^ h' /y/ 9 '"^ 0 ) 

T" = I 2 if 9 ' = h' = 0. 


if5'2 + /j'2 > 0 ^ 


We get 

N 2=(^1 Q ) , /" = Ve'2 0 > 0 , h" = y/ 9 '^ + h'^ > 0. 

If f" 0 h" = 0, i.e., N 2 = 0, from condition 17511 it follows that = 0, which is impossible because 
5'o n S' = {0}. Therefore, /" + h" > Q. Without loss of generality it can be assumed that /" 7 0 (otherwise 
one can take V3 = V5, v/ = vq, = V3, vg = V4). Therefore, we can assume /" = 1, taking iTi = f'vi, 
V 2 = f"v 2 , v/ = vr/f'/ Vs = Vs/f"- Keeping in mind that f" = 1, take the transformation 


T = Ti © 


i/Vi + T^h -h"/y/TTh^i2 \ ^ 
h"/Vl + h"^l2 l/y/lTh^h ) ^ ’ 


where Ty = y/T+h^l 2 . Then we obtain desired form for the submatrix N 2 . 

Condition m implies 

* * 

Ng = 

P 9 

\ -q P J 

Since the casep = q = 0 is impossible (the condition SqHS = {0}), we havep^Tg^ > 0. The transformation 
T = I 2 (B I 2 ®Ti (B I 2 , where 

_ f p/Vp'^ + g^' q/Vp'^ + g^ 


Ti = 


reduces Ng to the form 


' ' -q/V^^ p/V^+7 

\ 


Ng = 


/ * * 

* * 

p' 0 

V 0 p'; 


, p' = \/p2 0 > 0, 
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N.= 


retaining the submatrices Ni, N 2 , N 4 , and Nq. It follows from conditions of the i?-normality m and 
that 

/ sin^ 7 sin 7 cos 7 cos <5 \ 

— sin 7 cos 7 cos <5 
sin 7 cos 7 sin <5 0 

0 sin 7 cos 7 sin S J 


■ cos^ 7 




, 0 < 7 < tt/2, 0 < S < tt, 


N. = 


t 


sin^ 7 / 2/3 —t sin 7 cos 7 cos 5/2/3 + s 


At last, take transformation lISl) . where 
T 2 = 

and reduce N to the final form: 


0 0 s/(sin 7 COS 7 sin 5 ) </(sin 7 cos 7 sin 5) 

0 0 —t/(sin 7 cos 7 sin 5) s/(sin 7 COS 7 sin 5 ) 


f a (3 

—/3 a 

0 


N = 


0 
0 
0 
0 

V 0 


0 

0 

a 


0 

1 

/? 


—(3 a 
0 0 


0 

0 

0 

0 

a 


0 

0 

0 

0 

/3 


—j3 a 
0 0 


sin^ 7/2/3 

sin^ 7 

— sin 7 cos 7 cos 5 
sin 7 cos 7 sin 5 
0 
a 

-P 


0 0 

0 < 7 < 7 r/ 2 , 0 < 5 < TT. 


0 \ 
sin 7 cos 7 cos 5/2(3 
sin 7 cos 7 cos 5 
— cos^ 7 
0 

sin 7 cos 7 sin 5 

P 

a 


( 90 ) 


Due to Proposition 3 the matrix obtained is indecomposable. Let us check the iL-unitary invariance of 
the parameters 7 and 5. Suppose some iJ-unitary matrix T reduces the matrix N to the form N: 


where 


/ Ni 

N 2 

iVa \ 

1 

( Ni 

N 2 

N 3 \ 

N= 0 

N 4 

iVa 

, N = 

0 

N 4 

iVa 

V 0 

0 

iVi / 

\ 

V 0 

0 

Ni ) 


Ni = 


a P 

—/3 a 


0 0 0 0 
0 10 0 


/ 


N. = 


N4 = Ni © A^i, N2 = 

\ u 

sm 7 cos 7 cos 5 \ 


• 2 

sm 7 


N. = 


— sin 7 cos 7 cos 6 
sin 7 cos 7 sin 5 

0 

sin^ 7 

— sin 7 cos 7 cos 5 
sin 7 cos 7 sin 5 

0 


— cos^ 7 

0 

sin 7 cos 7 sin 5 J 

sin 7 cos 7 cos 5 \ 

— cos^ 7 

0 


sin 7 cos 7 sin 5 / 
0 < 7,7 < 7 r/ 2 , 0 < 5 , 5 < TT. 

Then, according to Proposition 2, T has the block triangular form 


T = 
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with respect to the decomposition i?® = 5'o-i-S'-i-S'i. Combining condition ll7^ NT = TN and ir7!?ll TT^*! = I, 
we get 

rp __rp _ ( cos 0 sin \ rp _rp ( COS ^/> sio \ 

^ ® \^ — sin (/) cos (j) ) ' ^ \^ — sin V' cos '0 j ’ 


where 


/ ^37 


To = 


ii3 


tl4 


-tl4 il3 + ^ 


ii5 ii6 
—^16 ^15 


T. = 


hs 


I ± sin d) 

^38 ^37 


^57 

\ —ha 


ha 

h7 


hi 

taa 

tai 

taa 


—ti 3 cos 20 — ti 4 sin 20 + 


—ti 3 sin 20 + ti 4 cos 20 — 


sin® 0 


cos 0 sm 


P 

-ti 5 cos (0 + 0 ) - tie sin (0 + 0 ), 
-ti 5 sin (0 + 0 ) + tie cos (0 + 0 ). 


Substituting the expressions for T 4 , Ts, Te in the formula -/V 4 T 5 + iVsTe = T 4 N 3 + T 3 N 1 which follows from 
GHJ, we obtain 


iVs 


/ sin^ 7 

— sin 7 cos 7 cos <5 
sin 7 cos 7 sin <5 cos(0 — 0) 
COS 7 sin S sin(0 — 0) 
y — sin 7 cos 7 sin (5 sin (0 — 0 ) 


sin 7 cos 7 cos <5 
— cos^ 7 
sin 7 

sin 7 cos 7 sin <5 cos (0 


\ 

0 ) / 


hence 0 = 0 , hence 7 = 7 , <5 = d. 

Thus, we have proved that 

// an indecomposable H-normal operator acts in a space i?® of rank 2 and has 2 eigenvalues: a ± t/3 
(cy, P G 3?, /3 > 00 then the pair {N,H} is unitarily similar to the canonical pair { (^,123)}. 

We have considered all the possible alternatives for an indecomposable operator N and have obtained 
the canonical forms for each case. Thus, we have proved Theorem 2. 
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